UNCLASSIFIED 


ESTIMATION  OF  MULTIVARIATE  BINARV  DENSITV  USING  1/i 

ORTHONORMAL  FUNCTIONS  <U>  PITTSBURGH  UNIV  PA  CENTER 
FOR  MULTIVARIATE  ANALVSIS  X  R  CHEN  ET  AL  OEC  86 
TR-86-48  AFOSR-TR-87-1075  F49620-85-C-0008  F/G  12/1 


NL 


982  98iV 


ktnMW&i 


UI 


.  A*  4'-  oW  -! 


StCUHIfY  CLAV^IMI  A  llON  O*  f»M  HAoVflThwi  Umim  ttitmtmU) 


[I.  ftCPORT  NUft 


REPORT  DQCUHNfHlON  PAGE  «  « 

BCR  \  )  .  la.  GOVT  ACCESSION  HO.  ».  RECIPIENT'S  CATALOG  NUMBER 


07Q  \A  I  To  £i‘:> 


\  FILE  COPY 


4.  TrCl  (MU  Suftlltfloj 


>•  TYR«  Of  R  CROAT  ft  PCMOO  COVCRCO 

I  \J 

Technical  -  December  1986 


Estimation  of  multivariate  binary  density 
using  orthonormal  functions 


authors  • 

X.R.  Chen,  P.R.  Krishnaiah  and  W.Q.  Liang 


•  ■  PERFORMING  ORG.  REPORT  NUMNI 

86-48 _ 

a.  contract  or  grant  number! •>” 


F49620-85-C-0008 


performing  organization  name  and  ADDRESS 

Center  for  Multivariate  Analysis 

Fifth  Floor  Thackeray  Hall 

University  of  Pittsburgh,  Pittsburgh,  PA  1 52€ 

controlling  office  name  ano  address 

Air  Force  Office  of  Scientific  Research 

Department  of  the  Air  Force 

Bolling  Air  Force  Base,  DC  20332  ^  > 


10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  «  PORK  UNIT  NUMBERS 


r\ 

ia.  report  date 

December  1986 


II.  number  of  pages 

14 


14.  MONITORING  AGENCY  name  a  ADDRESSfff  dffforanl  from  ConlflUnt  Offi«o>  I  IS.  SECURITY  CLASS,  (of  ifilo  ifpofi) 


Unclassified 


Wo.  DECL  ASSlFlC ATIOn/ DOWNGRADING 
iCHCOULC 


I  (ft.  Ol  ST  HI  BU  TlON  STATEMENT  (ot  |M«  Htpott) 


Approved  for  public  release;  distribution  unlimited. 


IT.  DISTRIBUTION  statement  (of  Ifio  «a«fro< «  onloroKfn  Sfoca  to,  ff  Kffforonr  from  RoporU 


SEP2«t007 


IS.  supplementary  notes 


Key  words  and  phrases:  multivariate  binary  density,  orthonormal  system. 


shrinkage  estimate,  Walsh  functions. 


fio”  A&S1RACT  (C»«iflnwo  on  rcvcn#  ild o  If  noco«to#r  Idonillr  4lo«fc  numb9t) 


In  this  paper,  the  authors  studied  certain  properties  of  the  estimate 


of  Liang  and  Krishnaiah  (*7.  Multivariate  Anal.j  16,  162-172)  for  multivariate 


binary  density.  An  alternative  shrinkage  estimate  is  also  obtained.  The 


above  results  are  generalized  to  general  orthonormal  systems. 


00  1473 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (Iffion  Do lo  En foroUJ 


I 


R 


V.-jf  o'v'  o" -ry 


mmmsm 


M 


AFQSR.TR.  87-1075 


.".''"wT!  ^.: ;  r.f?^ 

l  '*"i.^.»'U^'"  .‘-  ‘  ’“V/; «r  i'fr,  '■'T?r:.v.*-.**'\'-~ '- "  '“*%>•'  •  •••..»••'  ..'  '  > 

V«^-;  •*»,,*f$f<fc*>i*  V**  -Or.  —  •!  r^TV'  •*  •  ■, --••  ,  ..  .  •  ’.,  - ^ ^  •>.. ^  ^  #> 


s^xsttFs*** 

.Vr-:tf4?;Ss5S^  ~  -  “-•  ‘  7  V -*£'«. ,?5Sg  jfr  1 

•'  ■•.5'?rT '■-v';-;  .  ^  '  .*f,r-  ;/•  •  V^,-  _?:'•  ?• v,  ";  ":  ■■  '  *  V  fJ&  “v”'* '-.  * 


[‘•?H 

$ 


Center  for  Multivariate  Analysis 
University  of  Pittsburgh 


87  9  25  002 


'»V»^v^,ir)T«r>;»‘<;t,l,»,tT»%>r>, 


ESTIMATION  OF  MULTIVARIATE  BINARY 
DENSITY  USING  ORTHONORMAL  FUNCTIONS* 


X.R.  Chen,  P.R.  Krishnaiah  and  W.Q.  Liang 

Center  for  Multivariate  Analysis 
University  of  Pittsburgh 


December  1986 


Technical  Report  No.  86-48 


Center  for  Multivariate  Analysis 
Fifth  Floor  Thackeray  Hall 
University  of  Pittsburgh 
Pittsburgh,  PA  15260 


j  Accession  For 

NTIS  GRA&l 
DTIC  TAB 
Unannounced 
JustificatlonL- 


□ 


By - 

distribution/ 
Availability  Codes 
Avail  and/or 
iDlst  I  Special 


A:L 


Research  sponsored  by  the  Air  Force  Office  of  Scientific  Research  (AFOSC) 
under  Contract  F49620-85-C-0008.  The  United  States  Government  is  authorized 
to  reproduce  and  distribute  reprints  for  governmental  purposes  notwithstand¬ 
ing  any  copyright  notation  hereon. 


ESTIMATION  OF  MULTIVARIATE  BINARY 
DENSITY  USING  ORTHONORMAL  FUNCTIONS* 


X.R.  Chen,  P.R.  Krishnaiah  and  W.W.  Liang 

Center  for  Multivariate  Analysis 
University  of  Pittsburgh 

ABSTRACT 

In  this  paper,  the  authors  studied  certain  properties  of  the  estimate 
of  Liang  and  Krishnaiah  (j.  Multivariate  Anal.,  16,  162-172)  for  multivariate 
binary  density.  An  alternative  shrinkage  estimate  is  also  obtained.  The 
above  results  are  generalized  to  general  orthonormal  systems. 


Key  words  and  phrases:  multivariate  binary  density,  orthonormal  system, 
shrinkage  estimate,  Walsh  functions. 


Research  sponsored  by  the  Air  Force  Office  of  Scientific  Research  (AFSOC) 
under  Contract  F49620-85-C-0008.  The  United  States  Government  is  authorized 
to  reproduce  and  distribute  reprints  for  governmental  purposes  notwithstand¬ 
ing  any  copyright  notation  hereon. 


1 .  INTRODUCTION 


In  a  number  of  situations,  the  experimenter  is  confronted  with  the 
statistical  analysis  of  the  data  which  is  binary  in  nature.  For  example, 
one  may  be  interested  in  diagnosis  of  the  disease  on  the  basis  of  symptoms. 

The  reliability  of  complicated  systems  can  be  studied  by  examining  as  to 
whether  its  components  are  functioning  or  not.  In  image  processing,  a 
picture  is  classified  on  the  basis  of  two  grey  levels  like  white  and  black 
using  some  threshold  value.  We  may  assign  a  score  of  1  or  0  according  as  the 
grey  level  is  white  or  black  respectively.  So,  it  is  important  to  study 
the  problems  of  estimation  of  multivariate  binary  density. 

Cencov  [3]  expressed  continuous  multivariate  density  as  a  series  of 
orthonormal  functions.  Bahadur 1 [1}  expressed  the  multivariate  binary  density 
as  a  series.  Ott  and  Kronmal  [5]  expressed  the  density  as  a  series  involving 
Walsh  functions.  Liang  and  Krishnaiah  [4]  also  expressed  the  density  in  terms 
of  Walsh  functions  but  the  coefficients  in  their  series  are  different  from 
those  used  by  Ott  and  Kronmal.  The  present  paper  is  a  continuation  of  the  work 

done  by  Liang  and  Krishnaiah.  - 

In  Section  2,  we  established  the  necessary  and  sufficient  condition  for 
the  difference  between  the  mean  integrated  square  error  (MISE)  of  the  esti¬ 
mate  of  Liang  and  Krishnaiah  and  the  MISE  of  a  "natural"  shrinkage  estimator 

_2 

to  be  of  order  0(n  )  where  n  denotes  the  sample  size.  It  is  also  shown  that 

_2 

0(n  )  is  the  best  possible  rate  that  can  be  obtained.  In  Section  3,  a  new 

shrinkage  estimate  of  the  multivariate  binary  density  estimate  is  proposed. 

It  is  shown  that  the  difference  between  the  MISE  of  this  estimate  and  the 

_2 

"natural"  shrinkage  estimator  is  of  order  0 ( n  ).  The  results  of  Sections  2 
and  3  are  generalized  to  general  orthonormal  systems  in  Section  4. 


2 


2.  SHRINKAGE  ESTIMATES  BASED  ON  WALSH  FUNCTIONS 
Denote  by  R  the  sets 

R  =  (Up  ... ,  xd).  =  0  or  1,  i  =  1,  ...»  d)  (1) 

and  by  P  =  {p(x) }  the  family  of  probability  functions  on  R  :  P  includes 
ail  such  function  on  R  satisfying 

P ( x )  >_  0  for  x  e  R;  IxeRP(x)  =  1.  (2) 

The  system  of  Walsh  functions  {0r>  is  defined  by 

0r(x)  =  (-1 )r'X/2d/2,  x  e  R  :  r  e  R.  (3) 

Each  p(x)  e  P  can  be  expressed  as 

p(x)  *  IreRarMX)*  (4) 

So  the  estimation  of  p(x)  involves  that  of  {ar>. 

Suppose  that  X  is  a  random  vector  with  range  R  whose  distribution 
p ( x )  is  unknown,  and  X,,  ...,Xn  are  i id.  observations  of  X.  Then,  since 

ar=E0r(X),  r  e  R,  (5) 

we  obtain  the  moment  estimate  of  a  as 

r 

K=  X  0r(X.)/n.  (6) 

r  i=l  r  1 

As  argued  in  [4],  there  are  reasons  to  consider  the  "shrinkage  estimator" 

¥*>  *  WrWx)  <7> 

in  replacing  the  "natural"  estimator  p(x)  =  IreRar0r(x)  derived  directly 
from  (4)  and  (6).  As  was  shown  in  [4],  the  mean  integrated  square  error 


of  p^(x)  to  be  denoted  by  MISE  (px)>  attains  its  minimum  upon  choosing 


xr  =  na2/(2‘d  +  (n-  l)a2),  r  e  R. 


(8) 


tfs: 

■-y- 


Since  in  practice  we  do  not  know  {ar>,  the  constants  Ur>  need  to  be 
estimated  from  the  sample.  Liang  and  Krishnaiah  proposed  such  an 
estimate  in  [4]  via  some  iteration  considerations,  as  follows. 


0, 


if  a2  <  4(n  -  l)2“d/n2 


r  [[n  +  (n2  -  4(n  -  1  )2-da~2)1/2]/2(n  -  1 ) ,  otherwise. 

Denote 

px.(x)  -  W?arMx)- 

Liang  and  Krishnaiah  proved  the  following  theorem. 


(9) 


(10) 


THEOREM  ([4],  Theorem  2).  If  0  <  |aj  <  2"d/2  for  all  r,  then  ^ 


MISE(px*)  -  MISE(px)  =  0(n"2). 


(11) 


Here  we  improve  and  complete  this  result  as  follows: 

THEOREM  1.  1°.  The  necessary  and  sufficient  condition  for  (11)  to 

be  true  is  that 


a  f  0,  for  all  r  e  R. 
r 


(12) 


^  We  point  out  here  that  "0  <  |a  [  <  2"d//2  for  all  r"  should  be  corrected 
as  "0  <  | a  |  <  2"d/2  for  all  r  f  (0,  0,  ....  0)M,  since  a  =  2"d/2 


4 


m 


$3 


2°.  0(n"^)  is  the  best  rate  obtainable.  Suppose  that  a*  is  any 


estimate  of  ar  (based  on  Xj,  ...,Xn)  and  form  the  estimator  p*(x)  = 


T  oDa*0  (x).  Then  for  at  least  one  p(x)  e  P,  we  have 
reK  r  r 


lim  sup  n^| MISE(p*)  -  MISE(p  ) |  >  0. 

a 


Proof.  1°.  Since  0r  is  bounded  by  1  on  R,  and  0r(X1),  ...,0r(Xn) 


are  iid.  random  variables  with  mean  a  ,  by  Bennett  inequality  (see  [2]), 


we  have 


P(|ar-  ap|  >.  e)  <_  2  exp(- 


for  arbitrarily  e  >  0.  The  validity  of  (14)  does  not  depend  on  the  value 


assumed  by  a^.  Use  (14)  to  replace  (4-5)  of  [4],  and  the  rest  of  the 


proof  for  sufficiency  runs  exactly  along  the  same  way  as  in  [4]. 


To  prove  the  necessity  of  (12),  suppose  that  a  =  0  for  some  r  e  R. 

o 


Denote  by  R  the  set  { r :  r  e  R;  ar  t  0)7  Then  we  have 


MISE(p*)  -  MISE(p  )  -  E(x*  a  )2| 


rn  r  ' 
O  0 


i^r.RlE(4r-Vr)2-E(V-Xj02|‘ 


r  r  r 


In  the  proof  of  sufficiency,  we  actually  proved  that 


E(ar- Vr)2- E(ap- x*ap) 2|  =  0(n"2),  r  e  R. 


On  the  other  hand,  since  a  =  0,  we  have 

o 


Var(ar  )  =  2  /n. 
o 


Hence,  by  Central  Limit  Theorem,  we  obtain 


/n  a  — N(0,2^),  (n  -*■  ») 
o 


5 


r 


»v 


a 


I 

I 


a 


a 


i 


i 


■a 


£ 


s 

<s 
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From  (9)  we  see  that  x*  >  i  when  x*  f  0.  Therefore 

r  —  c.  r 


1 im  inf  P(x*  >  i)  >  lim  inf  P(a  2  >  4(n  -  1 )2'd/n2) 

rv+oo  r  —  c  —  rvx»  r  — 


*  1  -  -L  f2  e"t2/2dt  =  g  >  0. 


So,  when  n  is  large,  we  have 


P(XJ  >  1/2)  >  g/2. 
o 


Use  again  (17).  We  get  for  arbitrarily  given  e  >  0 


lim  P(  | a  |  >_  e//n)  =  1  - 
rvn=°  o  /2ir 


P?d/2  2 
£d  -td/2  A 
e  x  '*dt  =  h 


Choose  e  >  0  small  enough.  We  have  h£  >  1  -  g/3.  Therefore,  for  n 


sufficiently  large,  we  have 


>_  e2/ 4n(P(x*  >  \)  +  P(|ar  |  >  jz)  -  1) 


>  e2/4n{g/2  +  1  -  g/3  -  1 )  =  ge2/24n. 


From  (15),  (16)  and  (21),  we  see  that  (11)  cannot  be  true. 


2°.  To  prove  the  second  part  of  this  theorem,  we  proceed  to  show  that. 


even  in  the  case  of  d  =  1,  (13)  holds. 


In  the  case  of  d  =  1,  x  can  only  take  two  values,  0  and  1,  and  we  shall 


write  p(0)  =  p,  p(l)  =  1  -  p ( 0 )  =  q.  In  this  case,  we  have 


aQ  =  1//2,  =  (2p-  l)//2. 


6 


t+;+ 


Suppose  that  (ag,  a*)  is  an  estimate  of  (aQ,  a-j).  Then,  since  it  is  well- 
known  in  the  theory  of  point  estimation  that  the  estimate 

^  =  /2  Xn  -  1//2 

p 

is  admissible  under  quadratic  loss  L(d,  a-j)  =  ( d  -  a ^ )  .  It  follows  that 
there  exists  p  f  0,  1,  such  that 


Ep(a*-  ^1  )2  1  E(ai  -  ai)2  =  2pq/n. 


(23) 


2  2 

Here  \ ^  =  ( 2p  -  1 )  /[(2p-l)  +  4pq/n],  and  simple  manipulations  shows  that 


Ep(x1a1  -  a^2  =  (2p  -  l)22pq/n  +  0(n-2). 


(24) 


Here  aQ  =  1//2,  XQ  =  1 ,  0Q(O)  =  0Q(1)  =  1//2.  Hence,  Ep(x0a0-a0)2  =  0. 
From  this  and  (24),  we  see  that,  denoting  p*(x)  =  lj.=  1a^0r(x) , 

MISEp(p*)  -  MISEp(px)  >  [l-(2p-l)2]2pq/n  +  0(n"2). 

2 

Since  p  ?  0,  1 ,  [1  -  (2p  -  1 )  ]pq  >  0,  and  we  have 


limsup  n2| MISE  (p*)  -  MISEfp  ’ 

n-x»  p  p  a 


This  proves  (13). 


The  above  method  of  proof  can  be  extended  to  the  general  case  of  d 
without  difficulty.  We  need  only  consider  a  subset  of  P  with  the  form 

Pj  =  (P(x):  p ( x )  e  P,  p(x^)  =  p,  p ( x ^ 2 ^ )  =  1  -p,  0  <  p  <  1} 
where  X^,  X^  are  two  chosen  points  in  R  and  x^  f  x^. 


3.  A  NEW  SHRINKAGE  ESTIMATE 


We  see  from  Theorem  1  that  in  order  to  have  (11),  the  condition  (12) 
is  necessary.  In  this  section  we  shall  introduce  a  new  shrinkage  esti¬ 
mator,  for  which  (11)  holds  without  any  restriction  on  { a r) .  For  this 
purpose,  define 


x  - 
r 


na^/(2"d  +  (n-l)a2),  if  |ar|  >  n‘ 


otherwise 


1/3 


(25) 


and  the  shrinkage  estimator 

A  A  A 

pr(x)  =  y  Dx  a  0  (x). 
KX  AreR  r  r  r 


(26) 


THEOREM  2.  For  p~  defined  in  (26),  we  have 

MISE(p-)  -  MISE(px)  =  0(n~2).  (27) 

Proof.  The  proof  is  a  slight  modification  of  the  proof  of  Theorem  2 
in  [4].  First,  according  to  (4-3)  of  [4],  we  need  only  to  show  that 

E(ar-x;ar)2EUr-Xr)2  =  0(n'4)  (28) 

for  each  r  e  R,  where  x\  lies  between  X and  Xr.  Consider  separately 
two  cases. 

1°.  ar  =  0.  In  this  case,  using  Bennett  inequality  (14),  we  find  that 

P(xr  t  0)  =  P(|ar|  >  n'1/3)  <  2  exp(-  n1/3/4)  =  0(n“4). 

Since  X^  is  bounded  by  1 ,  we  get 

E(xr  -  xr)Z  =  ExJ  =  0{n"4) . 

As  a  ,  x',  a  are  bounded,  we  get  (28). 


2°.  ap  f  0.  Again,  using  Bennett  inequality,  we  have 

p(lar“^ri  1  iarl/2)  i  e"cn,  for  some  c  >  0.  (29) 

Put  fn(t)  =  nt/(2_c*  +  (n  -  1  )t).  There  exists  c-|  >  0  such  that  f ^ ( t )  <_ 
.1  2 

c-jn  for  t  |  a  ]  /4,  and  c-j  does  not  depend  on  n.  Hence 

I  VS*1  -  cin"1i^r'a?i' 

Since  E|a2  -  a2|2  <_  2E | -  a^ | ^  =  2  Var(ar)  =  0(n~S ,  we  have 

E(Ar-Xr)  =  0(n-3) .  (30) 

On  the  other  hand,  we  have 

E(ar- x;ar)2  <  2  Var(ap)  +  E(1  -  Ap2.  .  (31) 

We  have 

1-Xr=0(n_1)  (32) 

and 

1  -  xr  =  0(n_1).  when  |ap|  >_  |ar|/2.  (33) 

From  (32)-(33),  and  observing  that  X^  lies  between  X^  and  X^ ,  we  get 
in  view  of  (29) 

E(1  -x;)2  <  0(n~2)  +  P(|ar|  <  |ar|/2 

£  0(n"2)  +  P(|ar  -  ap|  >  |ar|/2) 

=  0(n"2) . 

Since  Var(ar)  =  0(n"S»  from  (31)  and  (34)  we  get 


(34) 


"J»« ■  ^  "J  ■*  ■  *  ■■  »_»■ 
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4.  ESTIMATES  BASED  ON  GENERAL  ORTHONORMAL  SYSTEM 

In  the  above  discussions  we  chose  Walsh  functions  as  our  orthonormal 
system.  There  are  infinitely  many  orthonormal  systems  which  can  be  chosen 
for  the  estimation  purpose.  The  particular  choice  depends  upon  convenience 
and  the  needs  for  applications.  For  instance,  the  following  system 
0r(xr)  =  1,  0r(x)  =  0  for  x  /  xr,  r  e  R 

leads  to  the  usual  frequency  estimate  and  its  shrinkage.  We  now  indicate 
that  the  results  of  Sections  2  and  3  are  still  valid  for  any  choice  of 
orthonormal  system  {0^} .  The  only  modification  is  that  in  the  definitions 
of  \r,  A*  and  Ar  in  (8),  (9)  and  (25),  2" ^  should  be  replaced  by 

br  =  £reRp(x)0?(x)-  (36) 


THEOREM  3.  Suppose  that  (0r >  is  any  orthonormal  function  system  on 

R,  then  the  conclusions  of  Theorem  2  and  Theorem  3  remain  valid  if  the 

definitions  of  A.  a*  and  A„  are  modified  as  stated  earlier, 
r  r  r 

The  proof  is  obvious,  since  in  the  proofs  of  Theorem  2  and  Theorem  3, 
the  special  form  of  Walsh  functions  plays  no  special  role,  except  one  point 
which  we  now  discuss. 

In  the  proof  of  Theorem  1.  2°,  we  made  use  of  the  fact  that  a^  =  ]//2 
is  a  constant,  so  that  AgiHg  gives  an  exact  estimation  of  ag.  In  the  general 
case  this  situation  no  longer  holds.  In  general  we  have 

/aA  /0O(O)  V°>\'  (  ”  \ 

ViJ  ’v>o(,>  ¥’y  V-pJ 

or 

aQ  =  ap  +  b,  a-|  =  cp  +  d 


for  some  known  constants  a,  b,  c,  d.  An  inspection  of  the  proof  of 


Theorem  1,  2°  convinces  us  that  we  have  to  prove  that  (aj^  +  b,  cXn  +d) 
is  an  admissible  estimate  of  (ag,  a-j)  under  quadratic  loss.  That  is  to 
say,  if  tnere  exists  an  estimate  (a^,  a|)  such  that 

Ep(a0'a0)2  +  Ep(al  "al)2  -  (a2  +  c2)pO  -p)/n  (37) 

for  all  p  e  (0,  1),  then  in  (37)  we  have  equality  for  each  p  e  (0,  1). 
For  a  proof  of  this,  we  put 

S0  =  (a*-b)/a,  =  (af-d)/c  (38) 

and  tg  =  a2/(a2  +  c2),  t-|  =  c2/(a2  +  c2).  Note  that  tg  >  0,  t-j  >0  and 
tg  +  t-j  =  1.  Owing  to  the  convexity  of  the  function  f(x)  =  x  ,  we  have 

(t0Cg  +  t1?1  -p)2  <  tgUg-p)2  +  t1(51  -P)2.  (39) 

Noticing  that 

Ep(50  -  P>2  =  VJ-W 

Ep(5,-P)2  -  Ep(a|-a,)2/c2 

and  (37) ,  we  have 

E(tgC0  +  t1c1  -p)2  1  (a2  + c2)[Ep(a*  -  aQ)2  +  Ep(a|-a1)2]  =  pq/n.  (40) 

But  is  an  admissible  estimate  for  p.  Hence,  we  must  have  equality  in 

_  2 

(40),  and  tgCg  +  t-jC-j  =  Xp.  Also,  since  x  is  strictly  convex,  in  order 
to  have  equality  in  (39),  we  must  have  Cg  =  .  Therefore  5q  =  ^  =  Xn, 

and  from  (38)  we  see  that  (a£j,  a*-|)  is  no  other  than  (aXn  +  b,  cXn  +  d), 
and  we  have  equality  in  (37)  for  each  p.  This  completes  the  proof  of 
Theorem  3. 
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